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Enhancement  of  Stochastic  Resonance  Using  Optimization  Theory  * 


Xingxing  Wu,  ^  Zhong-Ping  Jiang,  *  Daniel  W.  Repperger,  ^  and  Yi  Guo  ' 

Abstract.  The  traditional  stochastic  resonance  is  realized  by  adding  an  optimal  amount  of  noise,  while  the  parameter- 
tuning  stochastic  resonance  is  realized  by  optimally  tuning  the  system  parameters.  This  paper  reveals  the  possibility  to  further 
enhance  the  stochastic  resonance  effect  by  tuning  system  parameters  and  adding  noise  at  the  same  time  using  optimization 
theory.  The  further  improvement  of  the  maximal  normalized  power  norm  of  the  bistable  double-well  dynamic  system  with  white 
Gaussian  noise  input  can  be  converted  to  an  optimization  problem  with  constraints  on  system  parameters  and  noise  intensity, 
which  is  proven  to  have  one  and  only  one  local  maximum  for  the  Gaussian-distributed  weak  input  signal.  This  result  is  then 
extended  to  the  arbitrary  weak  input  signal  case.  For  the  purpose  of  practical  implementation,  a  fast-converging  optimization 
algorithm  to  search  the  optimal  system  parameters  and  noise  intensity  is  also  proposed.  Finally,  computer  sinmlalions  aic 
performed  to  verify  its  validity  and  demonstrate  its  potential  applications  in  signal  processing. 

Keywords!  Optimization,  Stochastic  Resonance,  Signal  Processing 

1.  Introduction.  Since  put  forward  by  Benzi  in  1981,  stochastic  resonance  has  been  increasingly 
attracting  the  interest  of  researchers.  It  is  a  phenomenon  of  certain  nonlinear  systems  in  which  the  syn¬ 
chronization  between  the  input  signal  and  the  noise  occurs  when  an  optimal  amount  of  additional  noise 
is  inserted  into  the  system.  In  this  case,  the  extra  noise  will  help,  rather  than  hinder,  the  performance 
improvement  of  the  system  by  maximizing  or  minimizing  the  chosen  performance  measure,  such  as  output 
signal-to-noise  ratio  (SNR),  or  mutual  information.  The  concept  of  stochastic  resonance  was  first  proposed 
to  address  the  problem  of  the  periodically  recurrent  ice  ages  [1].  Basically,  the  stochastic  resonance  has  four 
elements:  nonlinear  system,  information-carrying  input  signal,  noise,  and  performance  measure  [2].  Many 
kinds  of  nonlinccir  systems  have  demonstrated  the  stochastic  resonance  phenomena,  such  tis  static  systems 
[3],  dynamic  systems  [4][19],  discrete  systems  [21],  and  coupled  systems  [4][20].  The  traditional  stochastic 
resonance  requires  the  information-carrying  signal  to  be  weak  and  periodic.  Now,  aperiodic  and  suprathresh- 
old  signals  can  also  be  the  input  of  certain  stochastic  resonance  systems,  in  terms  of  aperiodic  stochastic 
resonance  (ASR)  [5]  and  suprathreshold  stochastic  resonance  (SSR)  [6]  respectively.  For  the  noise,  it  is  no 
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longer  limited  to  white  Gaussian  noise.  It  can  be  colored  [7],  or  non-Gaussian  noise  [8].  The  performance 
measure  is  adopted  here  in  order  to  quantify  the  stochastic  resonance  phenomenon  and  describe  it  more 
exactly.  For  the  periodic  stochastic  resonance,  the  output  signal-to-noise  ratio  [4]  is  the  most  commonly 
used  measure.  Other  measures,  such  as  cross-correlation  [9],  mutual  information  [10],  are  used  for  describing 
aperiodic  stochastic  resonance.  Over  the  years,  stochastic  resonance  has  found  applications  in  many  different 
areas,  such  as  in  physics,  chemistry,  biomedical  science,  and  engineering  [4][19].  The  noise  has  been  used 
to  enhance  the  tactile  sensation  [llj.  The  suprathreshold  stochastic  resonance  has  been  appliefi  to  cochlear 
implant  coding  [12].  One  of  the  important  applications  of  stochastic  resonance  is  in  signal  processing,  such  as 
signal  detection  [13][22],  signal  transmission  [14][15],  and  signal  estimation  [16].  For  a  stochastic  resonance 
system,  the  chosen  performance  measure  will  reach  its  maximum  or  minimum  when  the  synchronization 
between  input  signal  and  the  noise  occurs.  In  order  to  convert  noise  to  a  positive  factor,  the  stochastic 
resonance  system  should  be  adjusted  properly  to  maximize  or  minimize  the  performance  measure,  such  as 
the  signal-to-noise  ratio.  For  traditional  stochastic  resonance  systems,  this  is  realized  by  supplementing  the 
input  with  an  optimal  amount  of  additional  noise  [4].  Recently,  it  was  demonstrated  that  the  stochastic 
resonance  effect  can  also  be  realized  by  tuning  the.  system  parameters  to  their  optimal  values  without  adding 
noise.  This  is  called  parameter-tuning  stochastic  resonance  [17]  [23].  It  is  also  shown  that  tuning  system 
parameters  is  a  better  method  in  some  situations  than  adding  noise,  especially  when  the  initial  noise  inten¬ 
sity  is  already  beyond  the  resonance  region  [18].  Among  this  research,  either  noise  is  added,  or  the  system 
parameters  are  tuned,  but  not  both.  This  paper  will  focus  on  investigating  the  possibility  to  further  enhance 
the  stochastic  resonance  effect  by  tuning  system  parameters  and  adding  noise  at  the  same  time,  based  on  our 
initial  research  results  in  [26].  This  enhancement  will  cause  the  chosen  performance  measure  to  be  further 
increased  or  decreased  and  will,  in  turn,  have  the  potential  to  further  improve  the  system  performance. 

The  rest  of  the  paper  is  organized  as  follows.  In  Section  2,  the  normalized  power  norm  of  the  bistable 
double- well  d5mamic  system  with  Gaussian-distributed  weak  input  signal  is  proven  to  reach  a  higher  maxi¬ 
mum  by  tuning  system  parameters  and  adding  noise  at  the  same  time,  compared  with  that  by  either  tuning 
system  parameters  or  adding  noise.  This  result  will  be  extended  to  the  situation  with  an  arbitrary  weak 
input  signal  in  Section  3.  In  order  to  meet  the  high-speed  requirement  of  some  tasks.  Section  4  proposes  a 
fast-converging  optimization  algorithm  to  search  the  optimal  system  parameters  and  noise  intensity.  Section 
5  will  focus  on  verifying,  via  computer  simulations,  the  improvement  of  the  maximal  normalized  power  norm 
by  comparing  with  the  traditional  stochastic  resonance  and  parameter-tuning  stochastic  resonance.  The 
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potential  application  of  this  scheme  in  signal  processing  is  also  mentioned  in  this  section.  Finally,  Section  6 
concludes  the  paper  with  some  brief  remarks. 


2.  Enhancement  of  Stochastic  Resonance  with  Gaussian-Distributed  Weak  Input  Signals. 
In  traditional  stochastic  resonance  systems,  the  chosen  performance  measure  is  taken  as  a  function  of  the 
noise  intensity,  while  it  is  only  taken  as  a  function  of  the  system  parameters  by  fixing  the  noise  intensity 
at  its  initial  level  in  parameter-tuning  stochastic  resonance  systems.  In  some  stochastic  resonance  systems, 
the  performance  measure  is  affected  by  both  the  system  parameters  and  the  noise  intensity.  This,  however, 
does  not  necessarily  mean  that  it  is  possible  to  enhance  the  stochastic  resonance  effect  by  tuning  the  system 
parameters  and  adding  noise  at  the  same  time.  We  have  demonstrated  this  in  [26].  Now,  we  will  examine  the 
interesting  situation  where  the  stochastic  resonance  can  be  further  enhanced  by  tuning  the  system  parameters 
and  adding  noise  at  the  same  time. 

In  this  paper,  we  will  choose  the  nonlinear  bistable  double-well  dynamic  system  as  the  stochastic  reso¬ 
nance  system.  This  is  a  typical  stochastic  resonance  system  which  has  been  extensively  used  in  the  work  of 
others  [2][4][9][19].  As  shown  in  our  literature,  the  bistable  double-well  system  has  found  several  applications 
in  signal  processing.  In  [27],  it  is  used  to  amplify  the  coherent  signals.  As  a  nonlinear  filter,  it  is  also  used  to 
recover  the  multi-frequency  signals  corrupted  by  noise  [17].  Reducing  the  bit-error  rate  (HER)  of  the  binary 
signal  transmission  is  another  application  of  this  system  [28]. 

In  [9],  the  aperiodic  stochastic  resonance  (ASR)  was  demonstrated  in  the  following  nonlinear  bistable 
double-well  dynamic  system 


dx 

dt 


(1) 


where  !7(a;)  =  —[A  —  S{t)]x^/2  -f  x*/4  is  the  symmetric  potential  function  with  a  fluctuating  barrier.  A 
is  taken  as  a  positive  system  parameter  in  this  paper  and  is  used  to  shift  the  input  signal.  S{t)  is  the 
input  signal  with  zero-mean  average,  ^{t)  is  white  Gaussian  noise  with  zero  mean  and  an  autocorrelation  of 
=  2D5{t  —  s).  The  angular  brackets  denote  the  ensemble  average. 

The  performance  measures  chosen  to  describe  this  aperiodic  stochastic  resonance  are  cross-correlation 
measures  (power  norm  Cq  and  normalized  power  norm  Ci)  defined  as  follows  [9] 

Co  =  max{S{t)R{t  +  t)},  (2) 
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(3) 


_ Co 


where  S{t)  is  the  input  signal.  R{t)  is  the  system  response  characterized  by  mean  transition  rate  of  the 
system.  The  overbar  denotes  an  average  over  time,  t  is  a  time  lag. 

Usually,  there  is  no  explicit  expression  for  the  power  norm.  If  the  input  signal  is  weak,  (i.e.,  S{t)‘^  <  A^), 
and  is  a  Gaussian  distribution,  the  ensemble  averaged  power  norm  (Co)  and  the  ensemble  averaged  normalized 
power  norm  (Cl)  can  be  approximated  by  [9] 


(Co)  QoAoexp[-eo  +  AlS^{t)/2] 


,r  ) _ _ 

(exp[Ag52(t)]-l  +  a(D)Qo^exp[2eo-A2S2(t)l)i/2’ 

where 

c{D)=Ki{m),  iW))  -  Qoexp[-©o  +  Ag5W2], 
Qo=KQAIyf2Tt,  ©0  =  A^IAD,  Ao  =  A/2D. 


(5) 


According  to  [9],  the  parameter  Ko  is  used  to  account  for  normalization  factors  in  the  construction  of 
the  mean  transition  rate.  The  parameter  Ki  is  used  for  the  estimation  of  the  noise-induced  variance  cr{D). 
In  this  paper,  we  will  set  them  to  the  same  constant  values  as  in  [9],  i.e.,  Kq  =  1,  Ki  =  0.019. 

If  the  system  parameter  A  is  fixed,  the  normalized  power  norm  (Ci)  will  reach  its  maximal  value  when 
an  optimal  amount  of  noise  is  added  into  this  nonlinear  system  [9].  We  will  now  investigate  whether  the 
stochastic  resonance  effect  of  this  system  can  be  further  enhanced  by  tuning  the  system  parameters  and 
adding  noise  at  the  same  time.  This  is  in  fact  a  problem  of  checking  whether  the  optimization  problem 
of  maximizing  (Ci)  with  the  constraints  on  both  the  system  parameters  and  noise  intensity  lias  a  local 
maximizer.  The  computer  simulations,  however,  show  that  this  constrained  optimization  problem  has  no 
local  maximizer  for  some  weak  input  signals.  So  two  additional  parameters  are  introduced  into  the  system 
and  the  new  system  equation  becomes 

r^xit)  =  [A  -  S{t)]x{t)  ~^+  m,  (6) 


where  Tq  is  a  positive  system  parameter  affecting  the  system  response  time  and  Xi,  is  a  positive  system 
parameter  affecting  the  barrier  height  of  its  potential  function. 

For  the  system  described  by  (6),  its  potential  function  is 
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(7) 


Uix)  =  -[A-S{t)]^  +  ^^ 

■  If  Ta  =  1,  the  ensemble-averaged  escape  rate  is  expressed  as  [9] 

im)  ^  ^  (8) 

where  U  is  the  potential  function,  Xmin  is  one  of  the  local  minimizers  and  Xmax  is  the  local  maximizer  of 
the  potential  function. 

Using  the  method  of  [9],  similar  to  (5),  we  derive  the  approximation  expression  of  (C'l)  for  system  (6) 

A  Q 

(exp[A^s^]  —  1  +  KiQ-^expfcTaAQ  —  s^A^/2])^/^’ 


where 


Q  = 


KqA 

V^TaTf' 


e  =  =  cr„AQ,  A  =  s  =  152(t)]i/2,  c  =  s/2nl2K^. 


AD 


2D 


We  will  choose  the  normalized  power  norm  {C\)  as  the  objective  function  to  be  maximized,  because  it 
emphasizes  the  similarity  between  the  input  and  the  system  output  and  can  still  predict  its  real  shape  even 
when  the  noise  intensity  is  outside  its  validity  range  [9].  The  enhancement  of  stochastic  resonance  can  then 
be  converted  to  finding  the  local  maximizer  of  the  following  optimization  problem 

max  {Cl),  (10) 

subject  to:  ^(t)^  <>1^,  .^4  >  0,  Dq  <  D<  Di 

The  constraint  S{t)'^  A?  comes  from  the  requirement  of  weak  input  signals.  The  system  parameter  A 
is  positive.  We  assume  that  the  noise  cannot  be  removed  from  the  system,  so  the  noise  intensity  D  cannot 
be  less  than  its  initial  value  jDq-  Of  course,  it  cannot  be  arbitrarily  large  either. 

Both  A  and  D  will  be  taken  as  the  optimization  parameters  for  this  optimization  problem  (10),  while 
To  and  X\,  are  taken  as  the  supporting  parameters  to  ensure  this  constrained  optimization  problem  has  local 
maximizer  as  shown  later.  To  simplify  the  derivation  and  calculation,  the  direct  optimization  parameters  of 
(10)  are  A  and  Q,  from  which  the  values  of  A  and  D  can  be  calculated. 

In  order  to  prove  that  the  constrained  optimization  problem  (10)  has  a  local  maximizer,  we  will  first 
consider  the  following  unconstrained  optimization  problem 

max  {Cl).  (11) 


Proposition  1  The  first-order  necessary  condition  for  the  local  maximizer  of  unconstrained  optimization 
problem  (11)  has  one  and  only  one  solution  (Q*,  A*). 
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Figure  1:  /i(A)  and  -/2(A)  (a)  To  =  1,  s  =  0.1  (b)  Tu  =  1,  s  =  0.1 


Proof.  According  to  the  first-order  necessary  condition  for  the  local  maximizer  of  (11),  we  have 


d{Ci) 

dA 


=  0 


and 


djCi) 

dQ 


=  0. 


.(12) 


From  (12),  we  can  derive  Q~^  =  cToA,  and  also 

(2  -  2s^A^)exp[s^A^]  —2  +  cfaKi{A  -h  s^A®)exp[l  -  s^A^/2]  =  0. 

Let 

/(A)  =  (2  -  2s^A^)exp[s^A^]  -  2  -h  cTaKi{A  +  s^A®)exp[l  -  s^A^/2]. 

We  have 

f'{A)  =  (-4s^A  -  4s^A^)exp[s^A^]  +  CTaKi{l  +  2s^A^  -  s‘'A'*)exp[l  -  s^A^/2). 


(13) 

(14) 

(15) 


Prom  (14)  and  (15),  we  know  that  /(O)  =  0  and  f{+oo)  =  —00.  /  (A)  will  be  positive,  if  A  — »  0“*“.  Ba.sed 
on  these,  we  can  conclude  the  first-order  necessary  condition  has  at  least  one  positive  solution  (A*,  Q’). 

■  /i(A)  =  (2  -  2s2A2)exp[s2A2],  (16) 

/2(A)  =  CTaKiA{l  -f  s^A^)exp[  1  —  s^A^/2]  —  2.  (17) 

/i(A)  will  decrease  monotonically  to  —00  as  A  — ►  -l-oo,  starting  from  /i(0)  =  2,  /2(A)  will  first  increase 
from  -2,  and  then  decrease  to  zero.  From  the  special  characteristics  of  /i(A)  and  /2(A),  it  follows  readily 
that  the  first-order  necessary  condition  can  only  has  one  positive  solution. 

Changing  the  values  of  Tq  and  s  which  me  positive  can  only  affect  the  values  of  Q*  and  A*.  It  will  not 
affect  the  property  that  the  necessary  condition  has  one  and  only  one  solution.  This  can  be  shown  in  Fig.  1 
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to  Fig.2. 

Proposition  2  The  system  parameter  Ta  can  be  used  to  continuously  adjust  the  value  of  A*  satisfying  the 
first-order  necessary  condition  for  the  local  maximizer  of  (11). 

Proof.  /2(A),  but  not  /i(A),  is  affected  by  the  system  parameter  t„.  From  the  special  characteristics  of 
these  two  functions,  we  can  find  out  that  the  increase  of  Ta  value  will  also  increase  the  value  of  A*  satisfying 
(13).  If  Ta  is  getting  close  to  zero,  A*  will  also  approach  zero.  This  means  the  value  of  A*  can  be  adjusted 
continuously  by  changing  Ta-  This  completes  the  proof  of  this  proposition. 

Proposition  3  The  unconstrained  optimization  problem  (11)  has  one  and  only  one  local  maximizer  when 
the  input  is  weak  (s  -4^  1)  and  the  values  of  system  parameters  Ta  and  Xs  are  chosen  properly. 

Proof.  We  need  to  prove  that  the  only  solution  (A*,  Q*)  of  the  first-order  necessary  condition  will  also 
satisfy  the  second-order  sufficient  condition  for  a  local  maximizer,  that  is,  the  Hessian  matrix  is  negative 
definite  at  the  point  (A*,  Q*). 
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At  the  point  (A*,  Q*),  we  can  get 

52(Ci)  -2s3A*2[sA*exp[s2A*2]  +  cr^ATiC-i  +  ^)]exp[l  - 

5A2  "  (exp[s2A»2)  -  1  + /TiQ-iexpll  -  ^])3/2  ’  . 

92(Ci)_  -<?t^sKiA*‘^ 

“2(exp[s2A*2]  -  1  +  iTiQ^-iexpll  -  £A::^])3/2’ 

d^Ci)  d^Ci)  -cV^sATiA'^expll  -  ^] 

dAdQ  ■"  dQdA  ~  2(exp[s2A*2]  -  i  +  AriQ*-iexp[l  -  2^])3/2  ’ 

The  numerator  of  can  be  proven  to  be  negative  for  the  weak  input  signal  (s  <C  1),  i.e. 

-2s^A*^exp[s^A*^]  -  2crasA'iA''(--  H - - — )exp[l - - — ) 

Q 

=  -2exp[s^A*^]  +  2-  cTaKiA’ll  -  s  +  s^A*^(l  +  -)]exp[l - - — ] 

~  -2s^A*^exp[s^A*^]  <  0. 


Prom  (18),  (19),  and  (20),  it  follows  that  and  are  all  negative  at  A  =  A*,  and 


The  Hessian  matrix  of  the  optimization  problem  (11)  is  defined  as 

(m  » 

I® 


At  the  point  (A*,  Q*),  the  determinant  value  of  this  Hessian  matrix  is 


2exp[s^A*^]  -  2  +  CTaKijA*  -  +  s=^A*3)exp[l  -  ^ 

4(exp[s3A*^]  —  1  +  A'iQ*~iexp[l  - 


According  to  Proposition  2,  the  requirement  sA*^  '3>  1  can  be  satisfied  by  properly  adjusting  the  system 
parameter  Tq  value.  In  this  situation,  the  determinant  value  will  be  positive,  because  of 

2exp[s^A*^]  —  2  4-  cTaKi{A* - ^  +  s^A*®)exp[l  -  — ] 

~  2exp[s^A*^]  —  2  +  cToA'i(A*  +  s^A*®)exp[l  —  — ] 

=  2s^A*^exp[s^A^^]  >  0. 

Obviously,  its  denominator  is  positive.  Prom  the  standard  test  on  negative-definiteness  of  a  symmetric 
matrix,  it  follows  that  the  Hessian  matrix  is  negative  definite.  This  completes  the  proof  of  Proposition  3. 
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Proposition  4  The  constrained  optimization  problem  (10)  with  weak  input  signal  (s  -^1)  has  one  and  only 
one  local  maximizer,  if  the  system  parameters  Ta  and  Xi,  are  chosen  properly. 

Proof.  Prom  (9),  we  can  get  A*  =  2/A*.  The  constraint  (A*)'^  »  will  be  satisfied  if  s^(A*)^  <  4, 
Combined  with  the  requirements  s(A*)^  1  and  s  -C  1  used  for  the  derivation  of  the  above  propositions, 

all  the  constraints  on  the  system  parameter  will  be  met,  if  Ta  value  is  chosen  properly  such  that 

s  <  s^(A’)^  <  4.  (22) 

Also,  D*  will  be  greater  than  Dq  and  less  than  Di  for  the  properly  chosen  parameter  Xj,,  because  of 
D*  =  t„X|/A*2. 

From  these,  we  prove  that  the  one  and  only  one  local  maximizer  of  the  unconstrained  optimization 
problem  (11)  will  also  be  the  one  and  only  one  local  maximizer  of  the  constrained  optimization  problem  (10) 
for  the  weak  input  signal  (s  1),  when  the  values  of  Ta  and  Xt,  are  chosen  properly. 

Obviously,  the  only  local  maximizer  of  (10)  is  also  its  global  maximizer.  This  completes  the  proof  of  this 
proposition. 

According  to  the  above  propositions,  the  normalized  power  norm  (Ci)  of  the  bistable  double- well  system 
with  Gaussian-distributed  weak  input  signal  can  be  maximized  by  tuning  system  parameter  A  and  adding 
noise  at  the  same  time  and  will  reach  a  higher  maximal  value  compared  with  that  by  only  tuning  the  system 
parameter  or  by  only  adding  noise. 

3.  Enhancement  of  Stochastic  Resonance  with  Arbitrary  Weak  Input  Signals.  The  above 
propositions  are  derived  under  the  assumption  of  Gaussian-distributed  weak  input  signal.  Now,  we  will  piove 
that  all  these  propositions  can  be  extended  to  the  case  with  arbitrary  weak  input  signals. 

Under  the  condition  of  A^S(t)^  <?C  1,  the  normalized  power  norm  (Ci)  can  be  approximated  by 

/\_Q 

~  (A2s2  -f  +  ^(1  -  +  A^)exp[cr„QA])i/2 ' 

The  related  constrained  optimization  problem  is  then  changed  to 

max  (Cl),  (24) 

subject  to:  A  >  0,  <  A^,  A^s^  -Cl,  Dq  <  D  <  Di. 

To  prove  Proposition  1,  the  following  can  be  derived  from  the  first-order  necessary  condition  for  a  local 
maximizer  of  the  optimization  problem  (11)  with  the  new  (Ci)  expression  (23): 
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CTaAQ=l, 

-s^A^  +  CTacKiil  +  s^A^/2  -  3s^AV8)  =  0. 


(25) 

(26) 


From  this,  the  Proposition  1  and  Proposition  2  can  be  proven  in  the  similar  way  as  before. 

In  order  to  prove  Proposition  3,  the  following  can  be  determined  if  the  constraint  s^A*^  ^  1  is  satisfied 
by  adjusting  Ta  properly 


-cT^eKis^A*^  +  s^A’(-4  +  -  cT^eKis^ A*"^ {2  -  3s^A*V4)  <  0,  (27) 


and 


-s^AVs  +  (-1  +  s^A*V2)  <  0. 


(28) 


From  these,  ,  and  are  all  proven  to  be  negative  at  A  =  A*,  and  Q  =  Q*. 

For  (23),  the  numerator  of  its  Hessian  matrix  determinant  value  can  be  simplified,  at  A  =  A*  and 
Q  =  Q*,  as 

s4a*3(2  _  2s2a*2)  +  cT„eiCis‘‘A’‘‘(—  -  ^^^) 

4, .3  ,  Zcr^eKis^A*^  CT^eK,s^A*\  ,  s^A'^  ,  15cT„,e/r,s«A*« 

+(.s  A  + - - - - - )+— ;r-  + - - ■  (^9) 


If  s^A*^  <C  1,  this  numerator  value  will  be  positive.  Similarly,  Proposition  3  is  proven  to  hold  for  this 
optimization  problem  (24). 

The  constraint  <SC  will  be  satisfied  if  s^A*^  <C  1  is  met  by  properly  adjusting  t,;,  because  of 
A*  =  2/A*.  The  Proposition  4  is  hereby  proved  for  the  optimization  problem  (24). 


4.  Optimization  Algorithms.  The  constrained  optimization  problem  (24)  has  no  closed-form  solution. 
The  optimization  algorithm  should  be  developed  to  search  the  optimal  system  parameter  and  the  optimal 
noise  intensity.  In  order  to  meet  the  high-speed  requirement  of  some  tasks,  a  fast-converging  optimization 
algorithm  is  proposed  for  (24)  with  an  arbitrary  weak  input  signal  in  this  paper. 

Proposition  5  The  Newton’s  method  for  solving  nonlinear  equations  can  be  used  to  search  the  optimal 
parameters  of  the  constrained  optimization  problem  (24)  with  a  local  Q-quadratic  convergence,  if  the  initial 
value,  Ta  and  Xb  values  are  properly  chosen. 


Proof.  According  to  Proposition  4,  the  constrained  optimization  problem  (24)  has  one  and  only  one  local 
maximizer  (A*,  Q*)  which  satisfies  the  equations  (25)  and  (26),  if  To  and  Xb  values  are  chosen  properly.  In 
this  case,  the  optimal  parameters  (A*,  D*)  can  be  obtained  by  solving  the  nonlinear  equation  (26). 

Let 
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/3(A)  =  -5“ A*  +  cTacKiil  +  s'^^^12  -  Ss^AVs).  (30) 

V/3(A*)  will  be  non-singular  in  this  case.  According  to  the  standard  arguments  from  [25],  the  Newton's 
method  can  be  used  to  solve  (26)  with  a  local  Q-quadratic  convergence,  if  the  initial  value  Ao  is  sufficiently 
close  to  A*. 

Based  on  Proposition  5,  our  proposed  optimization  algorithm  will  first  estimate  the  initial  An  value  wliich 
will  be  sufficiently  close  to  its  optimal  value  A*,  then  the  standard  Newton  algorithm  will  be  called  to  search 
the  optimal  value.  The  algorithm  is  divided  into  two  cases.  In  the  first  case,  only  A  and  D  are  adjustable, 
while  the  system  parameters  Tq  and  Xb  are  given  beforehand.  Their  values  will  ensure  the  existence  of  the 
local  maximizer  for  the  constrained  optimization  problem  (24).  In  the  second  case,  the  system  parameters 
Ta  and  Xb  should  also  be  optimized. 

Case  1: 

In  order  to  estimate  the  initial  values  for  different  input  signals,  a  table  is  constructed  which  describes 
the  relationship  between  the  input  signal  average  amplitude  value  s  =  [5^(t))*^^  and  the  optimal  value  A* 
related  to  this  input.  In  order  to  construct  the  table,  we  can  first  select  a  series  of  signal  average  amplitude 
values.  Usually,  these  values  divide  the  signal  average  amplitude  range  evenly.  For  each  signal  average 
amplitude  s  value,  the  related  optimal  value  A*  satisfying  (26)  can  be  obtained  off-line  using  the  standard 
Newton  algorithm.  Then,  the  pair  (s.  A*)  can  be  inserted  into  the  table.  If  the  algorithm  is  implemented  in 
software,  the  table  can  be  represented  by  a  one-dimensional  array.  For  any  given  input,  the  related  initial 
value  Ao  can  then  be  estimated  based  on  the  information  provided  by  this  table,  such  as  using  interpolation. 
It  will  be  sufficiently  close  to  its  optimal  value  to  ensure  the  required  convergence  speed,  if  the  table  is 
constructed  properly.  Algorithm  1  is  the  optimization  algorithm  for  the  first  case. 

Algorithm  1; 

Step  1: 

Calculate  the  average  amplitude  s  =  [5'2(t)]^^^  for  the  given  input  signal; 

Step  2: 

Estimate  initial  Aq,  using  the  constructed  table; 

Step  3: 

Solve  (26)  using  standard  Newton  algorithm  [25]; 

Step 

Calculate  A*  and  D*,  then  stop. 
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Case  2: 


In  this  case,  system  parameters  Ta  and  Xt,  are  unknown  and  should  be  optimized  for  the  constrained 
optimization  problem  (24).  Here,  we  also  assume  TaX^  is  a  properly  chosen  constant  which  ensures  the 
existence  of  the  local  maximizer  of  (24).  The  equation  (23)  reveals  that  the  smaller  the  value  is,  the 
larger  the  (Ci)  will  be  if  other  parameters  are  fixed.  This  means  that  there  will  be  no  local  maximizer  for 
parameter  Tq.  Also,  a  smaller  To  value  will  generate  a  smaller  A*  satisfying  (26)  and  make  it  easier  to  satisfy 
the  requirement  of  1.  Too  small  To  value,  however,  will  make  it  difficult  to  satisfy  the  constraints 

on  noise  intensity,  because  of  D*  =  TaA^/A’^.  For  a  given  input  signal,  there  will  be  a  region  for  Ta  value 
in  which  the  constrained  optimization  problem  (24)  will  have  a  local  maximizer.  The  optimal  Tq  value  will 
take  the  extremum.  In  the  following  algorithm,  we  will  take  the  smallest  Tq  value  as  its  optimal  value,  which 
will  ensure  the  existence  of  the  local  maximizer. 

Similarly,  tables  will  be  constructed  to  increase  this  algorithm’s  convergence  speed.  Case  2  will  have 
two  tables.  The  first  one  describes  the  relationship  of  A*  with  input  signal  average  amplitude  s  and  system 
parameter  Tq.  The  second  one  describes  the  relationship  between  input  signal  average  amplitude  s  and  r* 
which  is  the  optimal  value  under  this  definition  related  to  this  input  signal.  For  the  first  table,  we  will  first 
select  a  series  of  (s,  Tq)  values.  For  each  pair  (s,  Ta),  the  related  A*  satisfying  (26)  can  be  calculated  off-line 
using  standard  Newton  algorithm.  Then  the  pair  (s,To,  A*)is  inserted  into  table  one.  For  the  second  table, 
a  series  of  input  average,  amplitude  values  should  also  be  chosen  first.  The  related  optimal  value  t’  can  be 
obtained  off-line.  The  pair  (s,t*)  is  then  inserted  into  table  two.  If  the  algorithm  is  implemented  in  software, 
table  one  can  be  represented  by  a  two-dimensional  array,  and  table  two  is  represented  by  a  one-diniensional 
array. 

For  a  given  input  signal,  the  optimal  t*  value  can  be  estimated  based  on  the  information  provided  by 
table  two.  It  will  then  be  used  to  estimate  Aq  from  table  one  which  will  be  used  as  the  initial  value  to  solve 
(26)  by  Newton  algorithm.  Both  of  the  estimations  can  be  performed  Using  related  interpolation  algorithms. 
The  optimal  values  of  A*,  A*  and  D*  will  then  be  evaluated  to  decide  how  value  should  be  changed  for 
next  loop.  If  A*^s^  1  or  D*  >  Do  is  not  satisfied,  Tq  value  will  then  be  decreased.  If  D*  <  Di  is  not  met, 

its  value  will  be  increased  instead.  The  detailed  algorithm  is  shown  in  Algorithm  2. 

Algorithm  2: 

Step  1: 

Calculate  s  —  [5^(t)]^'^^  ,  set  xq  =  0,  j/o  =  +oo; 

Step  2: 
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Estimate  r*,  take  it  as  the  initial  value  Tqo; 

Step  3: 

Estimate  the  initial  value  Aq; 

Step  4- 

Solve  (26)  using  standard  Newton  algorithm  [25]; 

Step  5: 

If  Tak  is  too  large: 

(xfc+i+yjb+i ) 

J/fc-fl —  Ta(fc4*l)  —  2 

else: 

yk~\-l  —  Vk-)  ^fc+1  ~  '^afc 
if  Vk+i  =  +oo: 

Ta{k+\)  =  2rafc 

else: 

7-a(A:+l)  =  (^fc+l  +yfc+l)/2 

Step  6: 

If  I'^a(/c4'l)  ”  '^ak\  ^  S' 

If  all  constraints  are  satisfied 

Calculate  A",  and  D*,  then  stop. 

else 

No  optimal  solution 

else: 

Go  back  to  Step  3 

In  the  above  algorithm,  the  condition  “ too  large ”  means  D  <  Di-  In  eiddition,  the  “  no  optimal  solution  ” 
problem  can  be  solved  by  adjusting  TaX^  value  properly. 

The  above  algorithms  are  used  to  search  the  optimal  values  of  system  parameter  A  and  noise  intensity  D 
at  the  same  time.  In  this  algorithm,  the  nonlinear  equation  (26)  should  be  solved  to  obtain  the  optimal  value 
A*.  It  will  have  no  overhead,  compared  with  the  algorithms  which  are  used  to  search  either  the  optimal 
value  of  system  parameter  A  or  the  optimal  value  of  the  noise  intensity  D.  If  only  the  noise  intensity  is 
adjustable  and  the  value  of  system  parameter  A  is  fixed,  the  optimization  algorithm  for  (24)  will  only  search 
the  optimal  value  D*  to  maximize  (Ci).  From  (9),  we  can  find  that  Q  will  be  constant  and  (Ci)  will  be 
the  function  of  A.  From  the  necessary  condition  for  a  local  maximizer  =  0,  we  can  get  a  nonlinear 
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Nofmallzsd  Power  N«m  <C^> 


Figure  3:  Normalized  Power  Norm  (Ci) 

equation  similar  to  (26).  The  optimal  value  A*  can  be  obtained  by  solving  this  equation.  From  A*,  we  can 
get  the  optimal  noise  intensity  £)*.  Similarly,  if  only  the  system  parameter  A  is  adjustable  and  the  noise 
intensity  D  is  fixed,  the  optimization  algorithm  for  (24)  will  only  search  the  optimal  value  A*  to  maximize 
the  (Cl).  Also  from  (9),  we  can  notice  that  (Ci)  will  be  the  function  of  Q.  From  the  necessary  condition  for 
a  local  maximizer  =  0,  we  can  also  get  a  nonlinear  equation  similar  to  (26)  from  which  the  optimal 
value  Q*  can  be  calculated.  The  optimal  value  A"  will  in  turn  be  obtained.  From  above  analysis,  we  can 
conclude  that  the  optimization  algorithm  for  searching  both  A*  and  D*  will  have  the  same  complexity  as 
that  for  searching  either  A*  or  D*.  The  only  difference  is  the  nonlinear  equation  which  needs  to  be  solved. 

5.  Simulations.  Computer  simulations  are  performed  to  verify  the  above  propositions  and  demonstrate 
the  improvement  of  the  maximal  normalized  power  norm  (Ci),  compared  with  that  by  only  tuning  the  system 
parameter  and  that  by  only  adding  noise.  Figure  3  shows  that  the  constrained  optimization  problem  (24) 
has  the  local  maximizer.  Figure  4  compares  the  maximal  (Ci)  reached  by  three  different  methods:  (1)  only 
adjusting  system  parameter  A  while  letting  D  =  0.1  (2)  only  adjusting  noise  intensity  D  while  letting  A  —  1 
(3)  tuning  system  parameter  A  and  adjusting  the  noise  intensity  D  at  the  same  time.  From  this  figure,  we 
can  see  the  enhancement  of  stochastic  resonance  effect  by  this  scheme. 

Also,  computer  simulations  are  performed  to  reveal  its  potential  application  in  weak  signal  recovery.  The 
enhancement  of  the  stochastic  resonance  effect  mentioned  above  means  the  similarity  between  the  input  signal 
and  the  output  of  the  bistable  double-well  dynamic  system  with  white  Gaussian  noise  input  is  enhanced. 
The  system  output  will  carry  more  information  about  the  weak  input  signal,  if  it  is  used  for  the  weak  signal 
recovery.  This  will  make  it  easier  to  recover  the  weak  input  signal  from  the  noisy  system  output.  Figure  5 
is  the  simulation  model.  In  this  model,  A,  o,  and  b  are  the  system  parameters  and  a  =  1/Ta,b  =  1/(t„,A(^). 
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Figure  4;  Comparison  of  Maximal  (Ci) 


Figure  5:  Simulation  Model 

The  noise  intensity  D  will  affect  the  output  of  the  White  Noise  block.  The  Constant  block  with  value  "shi  ft” 
is  used  to  shift  the  average  value  of  the  input  pulse  to  zero.  The  User-Defined  Functions  block  f{u)  is  used 
to  generate  x^.  Figure  6  is  the  original  weak  input  signal  and  the  white  noise.  The  system  outputs  under 
different  system  parameter  values  and  different  noise  intensity  are  shown  in  Figure  7.  From  this  simulation, 
it  is  obvious  that  the  similarity  between  input  and  output,  or  the  input  signal  information  carried  by  the 
system  output,  is  greatly  affected  by  the  choices  of  the  system  parameters  and  noise  intensity.  The  weak 
input  signal  can  be  better  recovered  from  the  noisy  system  output  when  the  system  parameter  values  and 
noise  intensity  are  chosen  properly. 

6.  Conclusion.  This  paper  explicitly  reveals  that  it  is  possible  to  further  enhance  the  stochastic 
resonance  effect  of  the  bistable  double-well  dynamic  system  with  white  Gaussian  noise  input  by  tuning  system 
parameters  and  adding  noise  at  the  same  time.  The  fast-converging  optimization  algorithm  introduced 
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Figure  6:  Input  Signal  and  Noise  (a)  input  signal  (b)  white  noise 


Figure  7:  System  Output  for  Different  Pareuneter  Values  (a)  original  input  signal  (b)  A=0.11,  a=100,  b=:0.5, 
D=0.01  (c)  A=l,  a=0.08,  b=3600,  D=0.01  (d)  A=l,  a=0.08,  b=3600,  D=1  (e)  A=l,  a=0.08  b=3600, 
D=0.00001  (f)  A=l,  a=l,  b=l,  D=0.01 

enables  this  scheme  to  be  applied  into  applications  with  high-speed  requirements.  The  enhancement  of 
stochastic  resonance  effect  means  the  enhancement  of  the  similarity  between  the  input  signal  and  system 
output.  This  will  help  the  weak  signal  recovery  from  the  noisy  system  output.  Our  future  work  will  be 
directed  at  extending  the  initial  results  of  the  applications  of  this  scheme  in  signal  processing. 
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